Imaging method and imaging apparatus

ABSTRACT

An imaging unit reads a signal out of an image pickup device to generate an image signal based on captured image. Signal processor generates image data from the image signal. Sampling analog audio signal allows audio data to be generated. Controller controls operations of the imaging unit and signal processor to alter a frame rate of the image data to a predetermined set frame rate and to generate associated information for indicating the set frame rate or the like. Transmitter combines the associated information with the image data and the audio data to transmit the combined ones as material data.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The invention relates to an imaging method and an imagingapparatus.

[0003] 2. Related Art

[0004] In generation of contents on audio and/or video for broadcasting,it has been often done to generate the contents altering motion speed ofa subject partially for only a set period, in order to obtain enhancedeffects the creator wishes to create.

[0005] In such the generation of contents altering the motion speed ofthe subject, a higher frame rate relevant to a reference frame rate isillustratively set to generate the contents, and then the contents arereproduced at the reference frame rate so that slow motion contents canbe produced. Alternatively, a lower frame rate relevant to a referenceframe rate is set to generate the contents, and then the contents arereproduced at the reference frame rate so that quick motion contents canbe produced. Further, adapting the frame rate to be set and the framerate at the reproduction allows the motion speed of the subject to befreely altered.

[0006] Thus, the creator creates the contents for broadcasting using notonly the contents generated at the reference frame rate but also thecontents of altered motion speed of the subject, in order to obtain theenhanced effects the creator wishes to create when the contents arereproduced at the reference frame rate.

[0007] It has been proposed in Japanese Patent Publication No.H11-177903 that the video camera allows time axis to be compressed orextended in order to generate the contents of altered motion speed ofthe subject by altering the frame rate as described above.

[0008] When the contents for broadcasting are generated using not onlythe contents generated at the reference frame rate but also the contentsof altered motion speed of the subject as material, it is not easilydetermined at editing which of frame rates each of the contents has beengenerated if merely the material image is displayed. Further, when itperforms the speed change on the contents of altered motion speed of thesubject, it is impossible to determine whether or not the speed changecan be performed without any deterioration in image quality of thedisplayed image. If a frame rate is set (hereinafter, referred to as,“set frame rate”) to 10 times the reference frame rate, the reproducedcontents are displayed one-tenth in a motion of subject if the contentsgenerated at the set frame rate are reproduced at the reference framerate. Here, it is conceivable that since the set frame rate is 10 timesthe reference frame rate when the creator wants to display the contentsone-fifth in a motion of subject, the speed change can be easilyperformed without any deterioration in image quality of the displayedimage if performing a frame-skipping for each frame. It, however, isimpossible to determine whether or not the speed change can be performedwithout any deterioration in image quality of the displayed image,easily based on merely the displayed image.

[0009] It is an object of the present invention to present an imagingmethod and an imaging apparatus wherein determining which of frame ratesthe imaging has been performed is easily allowed.

SUMMARY OF THE INVENTION

[0010] According to an aspect of the present invention, an imagingapparatus comprises imaging device for reading a signal captured out ofan image pickup device so as to generate an image signal based oncaptured image, and signal processor for generating image data based onthe image signal. The apparatus also comprises controller forcontrolling operation of at least one of the imaging device and thesignal processor to set a variable frame rate of the image data to a setframe rate, and generating associated information for indicating atleast the set frame rate. The apparatus further comprises transmitterfor combining the associated information with the image data to transmitthe combined ones.

[0011] According to another aspect of the present invention, an imagingmethod comprises the steps of generating image data based on a signalread out of an image pickup device, setting a variable frame rate of theimage data to a set frame rate, generating associated informationincluding frame rate information for indicating the set frame rate, andcombining the associated information with the image data to transmit thecombined ones.

[0012] In the present invention, when reading signal captured out of animage pickup device so as to generate image signal based on capturedimage, and generating image data based on the image signal, thecontroller controls operation of at least one of the imaging device andthe signal processor to set a variable frame rate of the image data to aset frame rate and generates associated information for indicating atleast the set frame rate. Thus, the transmitter combines the associatedinformation with the image data to transmit the combined ones. Based onthe associated information, determining which of frame rates the imaginghas been performed is easily allowed, and thus, the contents of alteredmotion speed of the subject can be reproduced.

[0013] Further, a variable frame rate of the image data is freely set toa set frame rate by means of altering a frequency of reading the signalcaptured out of the image pickup device, controlling the signalprocessor to perform frame-skipping, or altering a frequency of readingthe signal captured out of the image pickup device and controlling thesignal processor to add the image data on a frame basis.

[0014] Alternatively, according to the invention, a sub-frame number isadded to each of the frames of the set frame rate included within oneframe period of reference frame rate. The sub-frame number is includedin said associated information. This allows the frame-skipping to beeasily performed when the reproduction speed is changed by performingthe frame-skipping on frame basis.

[0015] Further, according to the present invention, the signal processorsamples analog audio signal to generate audio data and the controllercontrols a sampling frequency of the analog audio signal in the signalprocessor based on the set frame rate. Then, the transmitter combinesthe associated information with the image data and the audio data totransmit the combined ones. This allows the audio to be reproduced at avariable speed fitting the image.

[0016] Additionally, the imaging apparatus further comprises a signalrecording apparatus wherein the transmitter transmits a signal combiningthe associated information with the image data to the signal recordingapparatus recording the signal thus combined on recording medium.

[0017] The concluding portion of this specification particularly pointsout and directly claims the subject matter of the present invention.However those skill in the art will best understand both theorganization and method of operation of the invention, together withfurther advantages and objects thereof, by reading the remainingportions of the specification in view of the accompanying drawing(s)wherein like reference characters refer to like elements.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018]FIG. 1 is a diagram for illustrating a configuration ofcontents-providing system;

[0019]FIG. 2 is a block diagram showing a configuration of imagingapparatus;

[0020]FIG. 3 is a flowchart showing an offer operation of sub-framenumber;

[0021]FIG. 4 is an illustration illustrating SDI format;

[0022]FIG. 5 is an illustration illustrating SDTI format;

[0023]FIG. 6 is an illustration illustrating SDTI-CP format;

[0024]FIG. 7 is a block diagram showing a configuration of anotherimaging apparatus;

[0025]FIGS. 8A through 8E are diagrams showing relationship (part 1)between the image data and the associated information;

[0026]FIGS. 9A through 9E are diagrams showing relationship (part 2)between the image data and the associated information;

[0027]FIG. 10 is a block diagram showing a configuration ofsignal-recording apparatus;

[0028]FIGS. 11A through 11C are illustrations each showing GUIrepresentation at an editing operation; and

[0029]FIGS. 12A through 12J are illustrations showing a reproductionoperation of the image.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0030] Referring to the drawings, the invention will now be described indetail with reference to an embodiment according to the invention. FIG.1 shows an entire configuration of contents-providing system forproviding the contents such as the contents for audio and/or video. Animaging apparatus 10 generates image data having its frame rate variablyset and combines associated information including frame rate informationon the frame rate of the image data with this image data to supply asignal recording apparatus 20 or an editing apparatus 30 with thecombined ones as material data DTm. When the imaging apparatus 10 isprovided with an audio input apparatus 19, it generates audio data tosupply the signal recording apparatus 20 or the editing apparatus 30with the audio data and the associated information as material data DTm.

[0031] The signal recording apparatus 20 receives the material data DTmand records it on recording medium. It also reproduces the material dataDTm recorded on the recording medium and supplies the editing apparatus30 with the reproduced data. The imaging apparatus 10 and the signalrecording apparatus 20 may be composed as a combination camera/recordersystem, be combined together each other as so-called “dockable” typedcamera/recorder system, and be separately provided with them beingconnected through a cable.

[0032] The editing apparatus 30 receives the material data DTm from thesignal recording apparatus 20 or the imaging apparatus 10 and performsediting process on the material data DTm to produce image data and audiodata the editor wishes to create. The editing apparatus 30 combinesassociated information including frame rate information with the imagedata and the audio data thus produced to generate contents-data DC forbroadcasting, distribution, and the like.

[0033]FIG. 2 is a block diagram showing a configuration of the imagingapparatus 10. Imaging unit 12 receives incident light through imaginglens system 11 and an image pickup device such as charge coupled device(CCD) mounted within the imaging unit 12 captures subject image on itsimage surface. The image pickup device generates imaged charge of thesubject image though photoelectric transfer characteristics thereof. Theimaging unit 12 also reads out the imaged charge thus generated on thebasis of driving signal CR from a timing generator 142, which will bedescribed later, and generates imaged signal Sp having a frame rate thatcorresponds to the driving signal CR to supply camera processing circuit131 in signal processor 13 with the imaged signal Sp.

[0034] Based on timing signal CT received from the timing generator 142,the camera processing circuit 131 performs various signal processing attimings synchronized with the imaged signal Sp. Illustratively, such thesignal processing represents noise-reduction processing for reducingnoise components from the imaged signal Sp by means of correlated dualsampling or the like, transformation processing from the imaged signalSp thus noise-reduced to digital image data, clamp processing of theimage data, processing on shading correction and pilling-up ofdeficiency in the image pickup device, γ correction, contour enhancementprocessing, knee correction, and the like. Further, the cameraprocessing circuit 131 performs other various signal processing underthe process conditions based on the operation control signal CS receivedfrom imaging control circuit 141 of controller 14. Thus, the cameraprocessing circuit 131 generates image data DV thus obtained accordingto various signal processing and supplies transmitting unit 15 with theimage data DV.

[0035] The timing generator 142 of controller 14 generates drivingsignal CR corresponding to operation control signal CS from the imagingcontrol circuit 141 and supplies the imaging unit 12 with the drivingsignal CR, thereby allowing a reading-out frequency of imaged charge inthe imaging unit 12 to be altered. It, then, also controls a frame rateof the imaged signal Sp to be set to a set frame rate FRs based on anoperation signal PSa from user interface unit 16. For example, framefrequency, 59.94 Hz or 29.97 Hz in the case of NTSC system or framefrequency, 50 Hz or 25 Hz in the case of PAL system is set to the framefrequency of reference frame rate FRr and an operation such that the setframe rate FRs is set to the one k times the reference frame rate FRr iscarried out (k is positive value, not limited to an integer). If so, aframe rate of the imaged signal Sp is controlled to be equal to the onek times the reference frame rate FRr.

[0036] Alternatively, altering a period of reading-out pulse (sensorgate pulse) for transferring imaged charge accumulated on each pixelfrom the respective pixels in an image pickup device such as CCD totransfer unit or the like allows reading-out frequency of imaged chargeto be altered, thereby altering the frame rate.

[0037] It may be adapted for Common Data Rate (CDR) system in this case.When the CDR system is used, available frame rate may be altered, and aframe rate of the signal output from the CCD may be constant, thusmaking processed rate in the camera processing circuit 131 or the likeconstant.

[0038] This CDR system has been generally disclosed in PCT ApplicationNo. PCT/JP03/0051 filed on Jan. 22, 2003.

[0039] The timing generator 142 also generates timing signal CTsynchronized with the driving signal CR and supplies the cameraprocessing circuit 131 and audio signal processing circuit 132 with thetiming signal CT. The timing generator 142 further generates frame rateinformation DM-FRs indicating the set frame rate FRs that is frame rateof image data DV and supplies the transmitting unit 15 with the framerate information DM-FRs. The timing generator 142 also generatessub-frame number BN. This sub-frame number BN is identification numberfor identifying each frame included in each of the frame periods of thereference frame rate FRr when the set frame rate FRs is set to be higherthan the reference frame rate FRr. The timing generator 142 supplies thetransmitting unit 15 with this sub-frame number BN as frameidentification information DM-BN.

[0040]FIG. 3 is a flowchart showing an offer operation of sub-framenumber in the timing generator 142. The timing generator 142 performs afrequency division on oscillated signal of a predetermined frequency,for example, to set the set frame rate FRs so that the frame periods ofthe reference frame rate FRr and the set frame rate FRs may besynchronized with each other, thereby generating a frame referencetiming indicating a generation of the driving signal CR and a breakpointbetween the periods of frames of reference frame rate FRr based on theframe period of the set frame rate FRs.

[0041] At step ST1, the timing generator 142 determines whether or notthe frame reference timing is detected. If the frame reference timing isdetected, the process goes to step ST2. If no frame reference timing isdetected, the process goes back to the step ST1.

[0042] When the frame reference timing is detected at the step ST1, theprocess goes to the step ST2, as described above, where the timinggenerator 142 initializes the sub-frame number BN, thus setting thesub-frame number BN to an initial value, for example, 0. The processthen goes to step ST3.

[0043] At the step ST3, the timing generator 142 determines whether ornot the frame reference timing has been detected during a period of timestarting from detecting the frame reference timing and ending before oneframe of period of the set frame rate FRs has been passed. If no framereference timing is detected, the process goes to step ST4 where thetiming generator 142 adds 1 to the sub-frame number BN and then updatesthe added sub-frame number BN. The process then goes back to step ST3.Thus, when no frame reference timing has been detected before one frameof period of the set frame rate FRs has been passed, the sub-framenumber BN is allocated in sequence to the set frame rate FRs for each ofthe frames of periods thereof.

[0044] Thereafter, when the timing generator 142 detects the framereference timing before one frame of period of the set frame rate FRshas been passed, the process goes back to the step ST2 where thesub-frame number BN is initialized.

[0045] Every frame of period of the reference frame rate FRr, thesub-frame number BN may be added to the frame image of the set framerate FRs provided during a frame period of time.

[0046] Referring back to FIG. 2, the imaging control circuit 141 of thecontroller 14 is connected with the user interface unit 16. When, in theimaging apparatus 10, switching operation and alteration operation ofthe frame rate are performed, the user interface unit 16 generatesoperation signal PSa corresponding to these operations and supplies theimaging control circuit 141 with the operation control signal PS. Whenreceiving the operation signal PSa from external equipment such as aremote controller, not shown, the user interface unit 16 also suppliesthe imaging control circuit 141 with the operation signal PSa.

[0047] The imaging control circuit 141 generates operation controlsignal CS to allow the imaging apparatus 10 to be operated according tothe operation signal PSa based on the operation signal PSa received fromthe user interface unit 16. It also supplies the camera processingcircuit 131 and the timing generator 142 with the operation controlsignal CS.

[0048] The audio signal processing circuit 132 receives analog audiosignal Sin. The audio signal processing circuit 132 also performssampling process on the analog audio signal Sin based on timing signalCT received from the timing generator 142 to generate digital audio dataDA and supply the transmitting unit 15 with the digital audio data DA.

[0049] The transmitting unit 15 generates associated information DMincluding the frame rate information DM-FRs and the frame identificationinformation DM-BN and combines the image data DV and the audio data DAwith the associated information DM to generate material data DTm. Thetransmitting unit 15 supplies the signal recording apparatus 20 or theediting apparatus 30 with the material data DTm. The associated data DMmay include not only information on the set frame rate FRs and thesub-frame number BN but also information on imaged date and time, imagedcondition, imaged contents, and the like.

[0050] It is conceivable that as a way to combine the image data DV andthe audio data DA with the associated information, the associatedinformation DM is inserted into a data stream of image or a header ofthe data stream when the image data DV and the audio data DA arecompressed to generate the material data DTm as the data stream.

[0051] For example, when a moving picture compression system in MovingPicture Experts Group (MPEG) standardized as ISO (InternationalOrganization for Standardization)/IEC (International ElectrotechnicalCommission) 13818-2 is utilized, the associated information DM may beinserted into a region for extension or the like provided in a picturelayer, thereby allowing the associated information DM to be inserted ona picture basis. Alternatively, when a synchronization multiplexedsystem in MPEG standardized as ISO/IEC 13818-2 is utilized, theassociated information DM may be inserted into an optional field or thelike provided in a header in Packetized Elementary Stream (PES).

[0052] Further, it is conceivable that when SDI format standardized asSMPTE (Society of Motion Picture and Television Engineers) 259M,Television-10-Bit 4:2:2 Component and 4fsc Composite DigitalSignals-Serial Digital Interface, in order to transfer non-compressedimage and audio data, SDTI format standardized as SMPTE 305M,Television-Serial Data Transport Interface (SDTI), in order to transfercompressed image and audio data, or SDTI-CP format standardized as SMPTE326M, Television-SDTI Content Package Format (SDTI-CP) that obtained byfurther defining SDTI format is utilized, the associated information DMmay be inserted into a signal having each format as UMID datastandardized as SMPTE 330M, Television-Unique Material Identifier(UMID). In the present invention, a way to combine the image data DV andthe audio data DA with the associated information is not limited to theabove ones. Various ways to do so are conceivable.

[0053] UMID is an identifier for uniquely identifying material data suchas image data and audio data. UMID defines Basic UMID or Extended UMIDthat is identifier as being obtained by adding Signature Metadata toBasic UMID. Basic UMID includes a label for identifying digital data,information for indicating whether or not overwrite or editing isperformed on the material data, a number for distinguishing the materialdata from each other, and the like. Signature Metadata includesinformation on material-data-forming date and time, correctioninformation (information on difference in time) concerningmaterial-data-forming time, location information indicative oflatitudes, longitudinal, and altitude, information on name oforganization, and the like. Signature Metadata also includes a regionfor user code, to which the associated information DM may be stored, forexample.

[0054] The Extended UMID thus obtained is inserted into each of thesignals having various formats. When SDI format is utilized, theExtended UMID is inserted into an ancillary data region, as shown inFIG. 4. When SDTI format is utilized, the Extended UMID is inserted intoan ancillary data region except for a header data portion, as shown inFIG. 5. When SDTI-CP format is utilized, the data to be inserted isinserted into a payload region on an item basis. Specifically, as shownin FIG. 6, system item composed of information on image and audio,picture item composed of image data, audio item composed of audio data,and auxiliary (AUX) item composed of other data are inserted insequence. In the system item, regions for metadata sets such as PackageMetadata set, Picture Metadata set, audio Metadata set, and AuxiliaryMetadata set are provided to allow the metadata to be insertedthereinto. This permits the extended UMID to be inserted into theseregions.

[0055] The above imaging apparatus 10 alters read-out frequency ofimaged charge in the imaging unit 12 to generate the material data DTmof a predetermined set frame rate FRs, thereby allowing the set framerate FRs to be successively altered. If, however, the set frame rate FRsis altered solely in a stepwise, frame-skipping allows the material dataDTm of a predetermined set frame rate FRs to be generated. This is,generating image data DVa having a constant frame rate higher than theset frame rate FRs and extracting image data of only the set frame rateFRs from the image data DVa allows the material data DTm of apredetermined set frame rate FRs to be generated.

[0056]FIG. 7 shows a configuration of another imaging apparatus 10 aperforming such a generation. In FIG. 7, like reference numbers refer tolike elements shown in FIG. 2 and thus, the detailed description thereofis omitted.

[0057] Timing generator 182 in controller 18 generates driving signalCRa corresponding to the highest value of the set frame rate FRs that isset through user interface unit 16 and supplies the imaging unit 12 withthe driving signal CRa. Based on the driving signal CRa, the imagingunit 12 generates imaged signal Spa having a fixed frame rate FRq higherthan a reference frame rate. The imaging unit 12 then supplies cameraprocessing circuit 131 with the imaged signal Spa. When the set framerate FRs may be altered up to n times (where n is positive number) thereference frame rate FRr, the imaging unit 12 generates the imagedsignal Spa having a frame rate n times the reference frame rate FRr, andsupplies the camera processing circuit 131 with the imaged signal Spa.In other words, the imaging unit 12 generates the imaged signal Spahaving a fixed frame rate without any influence of the set frame rateFRs that is set through the user interface unit 16.

[0058] The timing generator 182 also generates timing signal CTasynchronized with the driving signal CRa and supplies the cameraprocessing circuit 131 and audio signal processing circuit 132 in signalprocessor 17 and effective frame signal generation circuit 183 in thecontroller 18 with the timing signal CTa.

[0059] The camera processing circuit 131 generates image data DVa offixed frame rate FRq based on the imaged signal Spa and supplies theeffective data selection circuit 171 with the image data DVa. The audiosignal processing circuit 132 carries out a sampling based on the timingsignal CTa of a fixed frequency to generate audio data DAa and suppliesthe effective data selection circuit 171 with the audio data DAa.

[0060] Imaging control circuit 181 generates a set information signal CFindicating a set frame rate FRs based on operational signal PSa receivedfrom the user interface unit 16 and supplies the effective frame signalgeneration circuit 183 with the set information signal CF.

[0061] The effective frame signal generation circuit 183 extracts dataon a frame basis from the image data DVa based on a ratio of the fixedvalue of frame rate FRq of the image data DVa to a value of the setframe rate FRs indicated in the set information signal CF and generatesextraction control signal CC for generating image data DV of the setframe rate FRs. The effective frame signal generation circuit 183 alsosynchronizes the extraction signal CC with the timing signal CTa andsupplies the effective data selection circuit 171 with the synchronizedextraction control signal CC. When the frame rate FRq of the image dataDVa is n times the reference frame rate FRr and the set frame rate FRsis n/2 times the reference frame rate FRr, the effective frame signalgeneration circuit 183 generates the extraction control signal CC forcontrolling data extraction on a frame basis every other frame from theimage data DVa and supplies the effective data selection circuit 171with the extraction control signal CC synchronized with the timingsignal CTa.

[0062] The effective frame signal generation circuit 183 furthergenerates frame rate information DM-FRs indicating the set frame rateFRs based on the set information signal CF and supplies the transmittingunit 15 with the frame rate information DM-FRS. Since the effectiveframe signal generation circuit 183 may determine a number of frameduring a period of frame with the reference frame rate FRr according tothe extraction control signal CC, the effective frame signal generationcircuit 183 sets sub-frame number on a frame during a period of eachframe with the reference frame rate FRr. The effective frame signalgeneration circuit 183 then supplies the transmitting unit 15 with thissub-frame number BN as the frame identification information DM-BN.

[0063] The effective data selection circuit 171 extracts the image dataDVa and the audio data DAa of the frame indicated by the extractioncontrol signal CC and supplies the transmitting unit 15 with them as theimage data DV and the audio data DA. It is conceivable that theeffective frame signal generation circuit 183 may supply the effectivedata selection circuit 171 with the frame rate information DM-FRsindicating the set frame rate FRs, and the effective data selectioncircuit 171 may perform frame-skipping on the audio data DAa accordingto a ratio of the set frame rate FRs to a frame rate when the audio dataDAa is generated. When the frame rate FRq in generating the audio dataDAa is n times the reference frame rate FRr and the set frame rate FRsis n/2 times the reference frame rate FRr, the effective data selectioncircuit 171 performs the frame-skipping on the audio data DAa everyother sample. In this case, since an interval of the frame-skipping maybe shortened as compared by a case where performing the frame-skippingon the audio data DAa on a frame basis, audio having an excellent soundquality may be get based on the audio data DA.

[0064] Thus, the image data DVa having a fixed frame frequency makesunnecessary the alteration of operation frequencies in the imaging unit12 and the camera processing circuit 131 of the signal processor 17.This allows configurations of the imaging unit 12 and the cameraprocessing circuit 131 to be made simpler. Since only the dataextraction on a frame basis from the image data DVa may generate imagedata DV of the set frame rate FRs, generating the image data DV of apredetermined set frame rate FRs from the image data Dva is easilyallowed.

[0065] When the imaging apparatus is provided with video memory or anadder and a divider, it may generate the image data DV by adding theimage data every the predetermined frames. This allows a variable rangeof frame rate in the imaged signal Sp to be limited and the frame rateFRs to be successively altered. Adding the imaged signal Sp of n framesand dividing the signal level by n allows the signal having a frame rate1/n times the imaged signal Sp to be obtained even if a frame rate ofthe imaged signal Sp is not divided by n. Further, successively alteringthe read-out frequency of imaged charge allows the set frame rate FRs tobe successively altered.

[0066]FIGS. 8A through 8E and 9A through 9E are diagrams showingrelationship between the image data DV generated by the imagingapparatus 10, 10 a and the associated information DM.

[0067] When the set frame rate FRs is set to the one equal to or twicethe reference frame rate FRr as shown in FIG. 8A, the associatedinformation DM that includes the frame rate information DM-FRsindicating the set frame rate FRs, as shown in FIG. 8C, and the frameidentification information DM-BN indicating the sub-frame number BN, asshown in FIG. 8D is combined with image data DV as shown in FIG. 8B (inFIG. 8B, frame images based on the image data DV are shown). FIG. 8Eillustrates a relationship between time and the frame images. The framerate information DM-FRs may include a magnification of the set framerate FRs to the reference frame rate FRr in addition to the set framerate FRs. The frame rate information DM-FRs is indicated by themagnification in FIGS. 8E, 9C, and 12B.

[0068] When the set frame rate FRs is set to the one equal to or half asmuch as the reference frame rate FRr as shown in FIG. 9A, the associatedinformation DM that includes the frame rate information DM-FRsindicating the set frame rate FRs, as shown in FIG. 9C, and the frameidentification information DM-BN indicating the sub-frame number BN, asshown in FIG. 9D is combined with image data DV as shown in FIG. 9B (inFIG. 9B, frame images based on the image data DV are shown). FIG. 9Eillustrates a relationship between time and the frame images.

[0069]FIG. 10 shows a configuration of signal-recording apparatus 20such as a videotape recorder.

[0070] When the material data DTm that combines main data indicatingimage and/or audio with the associated information including frame rateinformation of the main data is input, encoder 211 of signal-recordingprocessor 21 receives the material data DTm as recording data. Theencoder 211 produces error-correcting codes and performs data-shuffling,multiplexing, channel coding, and the like based on operation controlsignal CIM received from a recording/reproducing control unit 24 usingthe material data DTm thus received so that the encoder 211 can generaterecording signal SW. The encoder 211 then supplies a terminal 212 a of achangeover switch 212 with the generated recording signal SW.

[0071] The recording/reproducing control unit 24 supplies the changeoverswitch 212 with a switch control signal CTW. Operations of thechangeover switch 212 are controlled based on the switch control signalCTW and thus, the recording signal SW is supplied to recording amplifier213 a or 213 b through terminal 212 b or 212 c of the changeover switch212.

[0072] The recording amplifier 213 a receives and amplifies therecording signal SW and supplies signal-switching device 221 a of signalrecording/reproducing unit 22 with the amplified signal SW. Therecording amplifier 213 b receives and amplifies the recording signal SWand supplies signal-switching device 221 b of the signalrecording/reproducing unit 22 with the amplified signal SW.

[0073] When recording a signal, the signal-switching device 221 asupplies magnetic head 222 a with the amplified recording signal SWreceived from the recording amplifier 213 a based on a changeovercontrol signal CWR received from the recording/reproducing control unit24. When reproducing a signal, the signal obtained by the magnetic head222 a is supplied to reproducing amplifier 231 a of the signalreproduction processor 23. Similarly, when recording a signal, thesignal-switching device 221 b supplies magnetic head 222 b with theamplified recording signal SW received from the recording amplifiers 213b based on a changeover control signal CWR received from therecording/reproducing control unit 24. When reproducing a signal, thesignal obtained by the magnetic head 222 b is supplied to reproducingamplifier 231 b of the signal reproduction processor 23.

[0074] Thus, supplying the magnetic heads 222 a, 222 b built in arotating head drum (not shown) with the amplified recording signal SWallows image data and audio data, and the associated information to berecorded on magnetic tape, which is not shown. The magnetic heads 222 a,222 b read out the signal recorded on the magnetic tape and supplies thereproducing amplifiers 231 a, 231 b, respectively, with the read-outsignal.

[0075] The reproducing amplifier 231 a receives and amplifies a signalobtained by the magnetic head 222 a and supplies a terminal 232 a of achangeover switch 232 with the amplified signal. The reproducingamplifier 231 b receives and amplifies a signal obtained by the magnetichead 222 b and supplies a terminal 232 b of the changeover switch 232with the amplified signal. A terminal 232 c of the changeover switch 232is connected with a decoder 233.

[0076] The recording/reproducing control unit 24 supplies the changeoverswitch 232 with a switch control signal CTR. Operations of thechangeover switch 232 are controlled based on the switch control signalCTR on which the signals output from the recording amplifiers 213 a, 213b, respectively, are selected. The selected signal is supplied to thedecoder 233 as reproduced signal SR. The decoder 233 decodes thereproduced signal SR and performs data separation, de-shuffling, anerror-correcting processing, and the like based on operation controlsignal CIM received from the recording/reproducing control unit 24.Thus, the decoder 233 generates the material data DTm that combines maindata indicating image and/or audio with the associated informationincluding frame rate information of the main data and then transmits thecombined ones.

[0077] The recording/reproducing control unit 24 is connected withinterface unit 25. The recording/reproducing control unit 24 receivesoperational signal PSv from the interface unit 25 or another operationalsignal PSw from an external apparatus such as the editing apparatus 30.The recording/reproducing control unit 24 also generates the switchcontrol signals CTW, CTR and the operation control signals CTM, CTMcontrolling operations of the encoder 211 and the decoder 233,respectively, based on the operational signal PSv or PSw. Therecording/reproducing control unit 24 further performs a driving controlof a magnetic tape or a rotation head. The recording/reproducing controlunit 24 generates a display signal PH and supplies display unit 26 withit to display an operational condition of a video tape recorder, andvarious information and the like.

[0078] Although a case where the material data DTm is recorded on themagnetic tape has been shown in FIG. 10, the material data may berecorded on disk-like optical or magnetic recording medium orsignal-reproduction device using semiconductor storage element.

[0079] The signal-recording apparatus 20 is enough if the image data DVand audio data DA are combined with the associated information CM andthey are recorded or transmitted. Thus, the material data DTm to beinput is not limited to the one having the associated informationinserted into the data stream, header, or the like. The material datamay be supplied through different signal lines.

[0080] Combining the main data with the associated information DM allowsediting processing and variable speed reproduction to be easilyperformed using the data generated by the imaging apparatus 10, 10 a orthe data recorded on the recording medium by the signal-recordingapparatus 20.

[0081] During an editing process of the data combining the main datawith the associated information DM, a speed available for a reproductionmay be set based on the set frame rate FRs. This speed is set so that anatural reproduction image can be realized through a frame-skipping of apredetermined frame period or a frame repetition. If a multiple of theset frame rate FRs to the reference frame rate FRr is more than one,namely, FRs/FRr>1, the speed available for reproduction is set bysearching for the divisors of this multiple other than one andcalculating the reciprocals of the searched divisors. If the set framerate FRs is 10 times the reference frame rate FRr, the divisors of thismultiple, 10 other than one are of 2, 5, and 10 and thus, thereciprocals thereof are of ½, ⅕, and {fraction (1/10)}, respectively.Selection of one of the reciprocals of divisors allows the speedavailable for reproduction to be set to less than the same speed asnormal reproduction speed. When the speed available for reproduction notless than the same speed as the normal reproduction speed is set topositive integer multiples of the normal reproduction speed, theframe-skipping periods in each of the speeds available for reproductionmay be equal to each other.

[0082] If a multiple of the set frame rate FRs to the reference framerate FRr is less than one, namely, FRs/FRr<1, the speed available forreproduction is set by searching for the reciprocals of this multipleand calculating the divisors of the searched reciprocals other than oneor integer multiples of the searched reciprocals. If the set frame rateERs is ⅙ times the reference frame rate FRr, the divisors of thesearched reciprocals other than one are of 2, 3, and 6 and the integermultiples of the reciprocals are of 6, 12, 18, - - - , respectively.Selection of one of them allows the speed available for reproduction tobe set. This allows frame-skipping periods or numbers of the framerepetitions in each of the speeds available for reproduction may beequal to each other. The speed available for reproduction of less thanthe same speed as the normal reproduction speed is set to theone/positive multiples the normal reproduction speed. This allowsnumbers of the frame repetitions in each of the speeds available forreproduction to be equal to each other. Since, however, this isconcerned with a repetition of images, a lower limit of the speedavailable for reproduction may be set to the same speed as the normalreproduction speed.

[0083]FIGS. 11A through 11C illustrate GUI representations each for anediting operation using the associated information. For example, in eachof the GUI representations, a material administration browser window 401as material administration display is provided at an upper-left sidethereof; a story board window 402 as reproduction order display isprovided at an lower-left side thereof; a monitor viewer window 403 asreproduced image display for displaying an image before or after theediting process is provided at an upper-right side thereof; and a timeline window 404 as reproduction time order display is provided at alower-left side thereof. An operation control bar 405 is providedbetween the monitor viewer window 403 and the time line window 404.

[0084]FIG. 11A illustrates the GUI representation where the reproductionspeed is the same speed as the normal reproduction speed. FIG. 11Billustrates the GUI representation where the reproduction speed is atenth the normal reproduction speed. FIG. 11C illustrates the GUIrepresentation where the reproduction speed is twice as much as thenormal reproduction speed. Locations and sharps of the materialadministration browser window 401, the story board window 402, themonitor viewer window 403, the time line window 404, and the operationcontrol bar 405 are respectively illustrative ones and thus, of course,they are not limited to the above locations and sharps.

[0085] The material administration browser window 401 shows a list ofitems of the material data available for editing. For each item of thematerial data, a stamp view (a thumbnail view) for indicating a title, alength, contents of items of the stored material data is shown.

[0086] The story board window 402 serves as a working area for aproduction of contents. Arranging the items of material data on thereproduced order allows the contents to be produced.

[0087] The monitor viewer window 403 represents not only an image basedon the material data but also a location of reproduction and a variablespeed bar for indicating a variable range of reproduction speed.

[0088] The time line window 404 serves as a working area for aproduction of contents according to a more detailed method to allocateitems of the material data along a time axis.

[0089] The operation control bar 405 represents an operation key forreproducing the items of material data and the contents arranged on thetime line window 404.

[0090] When reproducing the selected material data, the set frame rateFRs is determined on the basis of the associated information DM, andthen, such a calculation that the reproduction speed FP multiplied bythe set frame rate FRs equals a determined value ED is performed.Reproduction process conditions are determined on the determined valueFD. When the reproduction speed is the same speed as the normalreproduction speed as shown in FIG. 11A and the set frame rate FRs is 10times the reference frame rate FRr, the reproduction speed FP multipliedby the set frame rate FRs equals the determined value FD as (1*10=10).

[0091]FIG. 12A illustrates images based on the image data DV where theset frame rate FRs is 10 times the reference frame rate FRr. FIG. 12Billustrates the frame rate information DM-FRs for indicating the setframe rate FRs of each of the frame images. FIG. 12C illustrates theframe identification information DM-BN for indicating the sub-framenumbers. FIG. 12D illustrates the absolute frame numbers AN of the frameimages. Since the set frame rate FRs is 10 times the reference framerate FRr, there are 10 frame images of the set frame rate FRs within oneframe period of the reference frame rate FRr, thereby repeating thesub-frame numbers of zero through nine.

[0092] When the determined value FD is set to 10, namely, FD=10, asshown in FIGS. 12E through 12G, the image signal Svm is generated usingevery 10 frames, namely, with the image data of nine frames beingskipped. This allows reproduced image having same speed as the normalreproduction speed to be represented on the monitor viewer 403 based onthe image signal Svm. FIG. 12E illustrates the frame identificationinformation DM-BN for indicating the sub-frame numbers. FIG. 12Fillustrates the absolute frame numbers AN of the frame images. FIG. 1illustrates frame images represented by the image signal Svm.

[0093] An operation is carried out such that an indicated width of aframe image representation region 404 a in the time line window 404 ofFIG. 11A is enlarged along a direction indicated by an arrow A or asliding cursor 404 d represented by a short bold line in a consoleindication 403 c for variable speeds in the monitor viewer 403 is slidon a direction indicated by an arrow B. In this case, a processing asthe slow motion of reproduction is performed. For example, when thespeed available for reproduction is set to {fraction (1/10)} times thenormal reproduction speed, a reproduction speed indication 403 b in themonitor viewer 403 is altered to “×0.1” as shown in FIG. 11B. At thesame time, the sliding cursor 404 d is slid to a cursor positionrepresenting a tenth times the normal reproduction speed in the consoleindication 403 c. Since the long reproduction time is required, theindicated width of the frame image representation region 404 a is alsoenlarged, as shown in FIG. 11B.

[0094] If the speed available for reproduction is set to {fraction(1/10)} times the normal reproduction speed as shown in FIG. 11B, thedetermined value FD is set to one, namely, FD=10*({fraction (1/10)})=1.When the determined value FD is set to one, namely, FD=1, the imagesignal Svm is generated using every FD=1 frame of the image data DV,namely, without any frame being skipped. This allows the reproducedimages having a tenth times the normal reproduction speed to berepresented on the monitor viewer 403 based on the image signal Svm, asshown in FIG. 12A.

[0095] An operation is carried out such that an indicated width of theframe image representation region 404 a in the time line window 404 ofFIG. 11A is shortened along a direction indicated by an arrow B or asliding cursor 404 d in the console indication 403 c is slid on adirection indicated by an arrow A in the monitor viewer 403. In thiscase, a processing as the quick motion of reproduction is performed. Forexample, when the speed available for reproduction is set to twice asmuch as the normal reproduction speed, a reproduction speed indication403 b in the monitor viewer 403 is altered to “×2.0” as shown in FIG.11C. At the same time, the sliding cursor 404 d is slid to a cursorposition representing twice of the normal reproduction speed in theconsole indication 403 c. Since the short reproduction time is required,the indicated width of the frame image representation region 404 a isshortened, as shown in FIG. 1C.

[0096] If the speed available for reproduction is set to twice of thenormal reproduction speed as shown in FIG. 11C, the determined value FDis set to 20, namely, FD=10*2=20. When the determined value ED is set to20, namely, FD=20, the image signal Svm is generated using every 20frames, namely, with the image data of nineteen frames being skipped.This allows the reproduced images having twice speed as much as thenormal reproduction speed to be represented on the monitor viewer 403based on the image signal Svm, as shown in FIGS. 12H through 12J. FIG.12H illustrates the frame identification information DM-BN forindicating the sub-frame numbers. FIG. 12I illustrates the absoluteframe numbers AN of the frame images. FIG. 12J illustrates frame imagesrepresented by the image signal Svm.

[0097] Thus, the material data of the set frame rate FRs is altered tovarious rates using the associated information EM and the alteredmaterial data is combined with each other, thereby allowingpredetermined contents-data for broadcasting, a distribution, and thelike to be easily generated. Further, when the edited image data iscombined with the corresponding associated information to generate thecontents-data, the user side performs the same processing as the oneperformed in the editing apparatus using the associated informationincluded in the contents-data, thereby allowing the reproduction speedof contents to be altered at the user side. For example, the set framerate FRs is set to the one higher than the reference frame rate FRr togenerate the material data such as sports event casting and then, usingthe material data, the contents-data including the associatedinformation is generated. This allows only a predetermined scene to beseen in a slow motion by the user with other scenes being seen in normalreproduction speed.

[0098] The invention has been described in detail with reference tospecific embodiments of the invention in the form of the imagingapparatus 10 having a communication function. It should be understoodthat the invention is not be limited to the embodiments and that theinvention can be applied equally well to other types of electronicapparatuses. While the foregoing specification has described preferredembodiment(s) of the present invention, one skilled in the art may makemany modifications to the preferred embodiment without departing fromthe invention in its broader aspects. The appended claims therefore areintended to cover all such modifications as fall within the true scopeand spirit of the invention.

What is claimed is:
 1. An imaging apparatus comprising: imaging devicefor reading a signal captured out of an image pickup device so as togenerate an image signal based on captured image; signal processor forgenerating image data based on said image signal; controller forcontrolling operation of at least one of said imaging device and saidsignal processor to set a variable frame rate of said image data to aset frame rate, and generating associated information for indicating atleast said set frame rate; and transmitter for combining said associatedinformation with said image data to transmit the combined ones.
 2. Theimaging apparatus according to claim 1, wherein said controller allowsfor setting the variable frame rate of said image data by means ofaltering a frequency of reading the signal captured out of said imagepickup device.
 3. The imaging apparatus according to claim 1, whereinsaid controller allows for setting the variable frame rate of said imagedata by means of controlling said signal processor to performframe-skipping.
 4. The imaging apparatus according to claim 1, whereinsaid controller allows for setting the variable frame rate of said imagedata by means of altering a frequency of reading the signal captured outof said image pickup device and controlling said signal processor to addsaid image data on a frame basis.
 5. The imaging apparatus according toclaim 1, wherein said controller allows for adding a sub-frame number toeach of the frames of said set frame rate included within one frameperiod of reference frame rate so as to include said sub-frame number insaid associated information.
 6. The imaging apparatus according to claim1, wherein said signal processor samples analog audio signal to generateaudio data; wherein said controller controls a sampling frequency ofsaid analog audio signal in said signal processor based on said setframe rate; and wherein said transmitter combines said associatedinformation with said image data and said audio data to transmit thecombined ones.
 7. The imaging apparatus according to claim 1, furthercomprising a signal recording apparatus, wherein said transmittertransmits a signal combining said associated information with said imagedata to said signal recording apparatus recording the signal thuscombined on recording medium.
 8. An imaging method comprising the stepsof: generating image data based on a signal read out of an image pickupdevice; setting a variable frame rate of said image data to a set framerate; generating associated information including frame rate informationfor indicating said set frame rate; and combining said associatedinformation with said image data to transmit the combined ones.
 9. Theimaging method according to claim 8, wherein, in said step of setting,the variable frame rate of said image data is set by means of altering afrequency of reading the signal out of said image pickup device.
 10. Theimaging method according to claim 8, wherein, in said step of setting,the variable frame rate of said image data is set by means of performingframe-skipping.
 11. The imaging method according to claim 8, wherein, insaid step of setting, the variable frame rate of said image data is setby means of altering a frequency of reading the signal out of said imagepickup device and controlling said signal processor to add said imagedata on a frame basis.
 12. The imaging method according to claim 8,wherein said associated information includes a sub-frame numberallocated to each of the frames of said set frame rate included withinone frame period of reference frame rate.
 13. The imaging methodaccording to claim 8, further comprising the steps of: sampling analogaudio signal to generate audio data; and controlling a samplingfrequency of said analog audio signal based on said set frame rate,wherein, in said combining and transmitting step, said associatedinformation is combined with said image data and said audio data totransmit the combined ones.
 14. The imaging method according to claim 8,wherein, in said combining and transmitting step, a signal combiningsaid associated information with said image data is transmitted tosignal recording apparatus recording the signal thus combined onrecording medium.